Decomposing word embedding with the capsule network

Word sense disambiguation tries to learn the appropriate of an ambiguous word in a given context. The existing pre-trained language methods and based on multi-embeddings did not explore power unsupervised embedding sufficiently. In this paper, we discuss capsule network-based approach, taking advantage capsule’s potential for recognizing highly overlapping features dealing with segmentation. We propose method decompose into context specific embedding, called CapsDecE2S. is fed network produce its multiple morpheme-like vectors, which are defined as basic semantic units meaning. With attention operations, CapsDecE2S integrates reconstruct vectors context-specific embedding. To train CapsDecE2S, matching training method. method, convert learning binary classification that explicitly learns relation between senses by label non-matching. was experimentally evaluated two tasks, i.e., disambiguation. Results public corpora Word-in-Context English all-words Sense Disambiguation show that, model achieves new state-of-the-art tasks. source code can be downloaded from Github page1 .